Parallelizing Windowed Stream Joins in a Shared-Nothing Cluster

The availability of large number of processing nodes in a parallel and distributed computing environment enables sophisticated real time processing over high speed data streams, as required by many emerging applications. Sliding window stream joins are among the most important operators in a stream processing system. In this paper, we consider the issue of parallelizing a sliding window stream join operator over a shared nothing cluster. We propose a framework, based on fixed or predefined communication pattern, to distribute the join processing loads over the shared-nothing cluster. We consider various overheads while scaling over a large number of nodes, and propose solution methodologies to cope with the issues. We implement the algorithm over a cluster using a message passing system, and present the experimental results showing the effectiveness of the join processing algorithm.Comment: 11 page

Processing Exact Results for Queries over Data Streams

In a growing number of information-processing applications, such as network-traffic monitoring, sensor networks, financial analysis, data mining for e-commerce, etc., data takes the form of continuous data streams rather than traditional stored databases/relational tuples. These applications have some common features like the need for real time analysis, huge volumes of data, and unpredictable and bursty arrivals of stream elements. In all of these applications, it is infeasible to process queries over data streams by loading the data into a traditional database management system (DBMS) or into main memory. Such an approach does not scale with high stream rates. As a consequence, systems that can manage streaming data have gained tremendous importance. The need to process a large number of continuous queries over bursty, high volume online data streams, potentially in real time, makes it imperative to design algorithms that should use limited resources. This dissertation focuses on processing exact results for join queries over high speed data streams using limited resources, and proposes several novel techniques for processing join queries incorporating secondary storages and non-dedicated computers. Existing approaches for stream joins either, (a) deal with memory limitations by shedding loads, and therefore can not produce exact or highly accurate results for the stream joins over data streams with time varying arrivals of stream tuples, or (b) suffer from large I/O-overheads due to random disk accesses. The proposed techniques exploit the high bandwidth of a disk subsystem by rendering the data access pattern largely sequential, eliminating small, random disk accesses. This dissertation proposes an I/O-efficient algorithm to process hybrid join queries, that join a fast, time varying or bursty data stream and a persistent disk relation. Such a hybrid join is the crux of a number of common transformations in an active data warehouse. Experimental results demonstrate that the proposed scheme reduces the response time in output results by exploiting spatio-temporal locality within the input stream, and minimizes disk overhead through disk-I/O amortization. The dissertation also proposes an algorithm to parallelize a stream join operator over a shared-nothing system. The proposed algorithm distributes the processing loads across a number of independent, non-dedicated nodes, based on a fixed or predefined communication pattern; dynamically maintains the degree of declustering in order to minimize communication and processing overheads; and presents mechanisms for reducing storage and communication overheads while scaling over a large number of nodes. We present experimental results showing the efficacy of the proposed algorithms

A study on the Clustering Properties of Radio-Selected sources in the Lockman Hole Region at 325 MHz

Studying the spatial distribution of extragalactic source populations is vital in understanding the matter distribution in the Universe. It also enables understanding the cosmological evolution of dark matter density fields and the relationship between dark matter and luminous matter. Clustering studies are also required for EoR foreground studies since it affects the relevant angular scales. This paper investigates the angular and spatial clustering properties and the bias parameter of radio-selected sources in the Lockman Hole field at 325 MHz. The data probes sources with fluxes $\gtrsim$0.3 mJy within a radius of 1.8$^\circ$ around the phase center of a $6^\circ \times 6^\circ$ mosaic. Based on their radio luminosity, the sources are classified into Active Galactic Nuclei (AGNs) and Star-Forming Galaxies (SFGs). Clustering and bias parameters are determined for the combined populations and the classified sources. The spatial correlation length and the bias of AGNs are greater than SFGs -- indicating that more massive haloes host the former. This study is the first reported estimate of the clustering property of sources at 325 MHz, intermediate between the preexisting studies at high and low-frequency bands. It also probes a well-studied deep field at an unexplored frequency with moderate depth and area. Clustering studies require such observations along different lines of sight, with various fields and data sets across frequencies to avoid cosmic variance and systematics. Thus, an extragalactic deep field has been studied in this work to contribute to this knowledge.Comment: 16 Pages, 10 Figures, submitted after minor revision to MNRA

Constraining extra dimensions using observations of black hole quasi-normal modes

The presence of extra dimensions generically modify the spacetime geometry of a rotating black hole, by adding an additional hair, besides the mass $M$ and the angular momentum $J$, known as the `tidal charge' parameter, $\beta$. In a braneworld scenario with one extra spatial dimension, the extra dimension is expected to manifest itself through -- (a) negative values of $\beta$, and (b) modified gravitational perturbations. This in turn would affect the quasi-normal modes of rotating black holes. We numerically solve the perturbed gravitational field equations using the continued fractions method and determine the quasi-normal mode spectra for the braneworld black hole. We find that increasingly negative values of $\beta$ correspond to a diminishing imaginary part of the quasi-normal mode, or equivalently, an increasing damping time. Using the publicly available data of the properties of the remnant black hole in the gravitational wave signal GW150914, we check for consistency between the predicted values (for a given $\beta$) of the frequency and damping time of the least-damped $\ell=2,m=2$ quasi-normal mode and measurements of these quantities using other independent techniques. We find that it is highly unlikely for the tidal charge, $\beta \lesssim -0.05$, providing a conservative limit on the tidal charge parameter. Implications and future directions are discussed.Comment: 11 pages, 2 figures, 1 table, Revised version accepted in EPJ

Deep uGMRT observations of the ELAIS-North 1 field: statistical properties of radio--infrared relations up to z∼z \sim2

Comprehending the radio--infrared (IR) relations of the faint extragalactic radio sources is important for using radio emission as a tracer of star-formation in high redshift ($z$) star-forming galaxies (SFGs). Using deep uGMRT observations of the ELAIS-N1 field in the 0.3--0.5\,GHz range, we study the statistical properties of the radio--IR relations and the variation of the `$q$-parameter' up to $z=2$ after broadly classifying the faint sources as SFGs and AGN. We find the dust temperature (\tdust) to increase with $z$. This gives rise to \qmir, measured at 24\,\upmum, to increase with $z$ as the peak of IR emission shifts towards shorter wavelengths, resulting in the largest scatter among different measures of $q$-parameters. \qfir measured at 70\,\upmum, and $q_{\rm TIR}$ using total-IR (TIR) emission are largely unaffected by \tdust. We observe strong, non-linear correlations between the radio luminosities at 0.4 and 1.4\,GHz with 70\,\upmum luminosity and TIR luminosity(\ltir). To assess the possible role of the radio-continuum spectrum in making the relations non-linear, for the first time we study them at high $z$ using integrated radio luminosity (\lrc) in the range 0.1--2\,GHz. In SFGs, the \lrc--\ltir relation remains non-linear with a slope of $1.07\pm0.02$, has a factor of 2 lower scatter compared to monochromatic radio luminosities, and \qtirbol decreases with $z$ as \qtirbol = (2.27 \pm 0.03)\,(1+z)^{-0.12 \pm 0.03}. A redshift variation of $q$ is a natural consequence of non-linearity. We suggest that a redshift evolution of magnetic field strengths and/or cosmic ray acceleration efficiency in high-$z$ SFGs could give rise to non-linear radio--IR relations.Comment: Accepted for publication in MNRAS (20 pages, 20 Figures and one Appendix

Effects of CME and CIR induced geomagnetic storms on low-latitude ionization over Indian longitudes in terms of neutral dynamics

This paper presents the response of the ionosphere during the intense geomagnetic storms of October 12-20, 2016 and May 26-31, 2017 which occurred during the declining phase of the solar cycle 24. Total Electron Content (TEC) from GPS measured at Indore, Calcutta and Siliguri having geomagnetic dips varying from 32.23{\deg}N, 32{\deg}N and 39.49{\deg}N respectively and at the International GNSS Service (IGS) stations at Lucknow (beyond anomaly crest), Hyderabad (between geomagnetic equator and northern crest of EIA) and Bangalore (near magnetic equator) in the Indian longitude zone have been used for the storms. Prominent peaks in diurnal maximum in excess of 20-45 TECU over the quiet time values were observed during the October 2016 storm at Lucknow, Indore, Hyderabad, Bangalore and 10-20 TECU for the May 2017 storm at Siliguri, Indore, Calcutta and Hyderabad. The GUVI images onboard TIMED spacecraft that measures the thermospheric O/N2 ratio, showed high values (O/N2 ratio of about 0.7) on October 16 when positive storm effects were observed compared to the other days during the storm period. The observed features have been explained in terms of the O/N2 ratio increase in the equatorial thermosphere, CIR-induced High Speed Solar Wind (HSSW) event for the October 2016 storm. The TEC enhancement has also been explained in terms of the Auroral Electrojet (AE), neutral wind values obtained from the Horizontal Wind Model (HWM14) and equatorial electrojet strength from magnetometer data for both October 2016 and May 2017 storms. These results are one of the first to be reported from the Indian longitude sector on influence of CME- and CIR-driven geomagnetic storms on TEC during the declining phase of solar cycle 24.Comment: 41 pages, 14 figures. Accepted for publication in Advances in Space Researc

Comparative studies of Ionospheric models with GNSS and NavIC over the Indian Longitudinal sector during geomagnetic activities

This paper presents the storm time comparative analysis of the performances of latest versions of global ionospheric models: International Reference Ionosphere (IRI) 2016, NeQuick 2 (NeQ) and the IRI extended to Plasmasphere (IRI-P) 2017 with respect to Navigation with Indian Constellation (NavIC) and Global Navigation Satellite System (GNSS) derived ionospheric Total Electron Content (TEC). The analysis is carried out under varying geomagnetic storm conditions during September 2017-November 2018, falling in the declining phase of solar cycle 24. TEC data from Indore, located near the northern crest of the Equatorial Ionization Anomaly (EIA) along with data obtained from the International GNSS Service (IGS) stations at Lucknow, located beyond the anomaly crest; Hyderabad, located between anomaly crest and magnetic equator and Bangalore, located near the magnetic equator have been analysed. The models generally overestimated during the storm periods with the exception of IRI-P, which matched (with an offset of about 3-5 TECU) with the enhancement observed on September 7, 2017 (during the strong storm of September 2017), from stations around the anomaly crest. No significant match was observed by the other two models. This match of IRI-P is attributed to the plasmaspheric contribution as well as the capability of assimilating measured TEC values into this model. In the present study, to the best of our knowledge, first comparisons of the empirical model derived TEC with NavIC and GNSS measurements from an anomaly crest location, combined with the IGS observations from the magnetic equator to locations beyond the anomaly crest, are conducted during geomagnetically disturbed conditions. Since NavIC satellites are at higher altitudes(~ 36000 km), the inclusion of NavIC data to the existing model could give better ionospheric predictions over the Indian subcontinent.Comment: 42 pages, 13 figures. Accepted for publication in Advances in Space Researc